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Abstract. Markov automata (MA) and weak bisimulation have been proposed 
by Eisentraut, Hermanns and Zhang in 1101 . In this paper we propose early and 
late semantics of Markovian automata, and then introduce early and late weak 
bisimulations correspondingly. We show that early weak bisimulation coincides 
with the weak bisimulation in 1101 . and late weak bisimulation is strictly coarser. 
Further, we extend our results to simulations. 



1 Introduction 

Compositional theories have become a foundation for developing effective techniques 
for analyzing stochastic systems, for example see 0141 for compositional minimization, 
and 11411 II for component based verification. Recently, Markov automata (MA) have 
been proposed in IflOl as a compositional behavioral model supporting both probabilis- 
tic transitions and exponentially distributed random delays. MA can be considered as a 
combination of probabilistic automata (PA) fTTI and interactive Markov chains (IMC) 
lfl2l . A PA is obtained by disallowing random delays, whereas an IMC is obtained by 
restricting to degenerative probabilistic transitions. 

As the main result in IfTUl , the authors have proposed the notion of weak bisimu- 
lation relation, which is shown to be congruent with respect to parallel composition. 
Moreover, the proposed weak bisimulation conservatively extends that for probabilistic 
automata 0151171 and IMCs lfl2l . However, as pointed out in the conclusion in IfTUl . 

"a good notion of equality is tightly linked to the practically relevant issue 
of constructing a small (quotient) model that contains all relevant information 
needed to analyze the system ". 

Indeed, an example is given in the conclusion illustrating that an even weaker version 
of weak bisimulation would be expected. 

In this paper we address this problem by proposing such a weaker bisimulation. 
We start with discussing the example presented in the conclusion of ifTOl . An extended 
version is shown in Fig.Q] where: 

- In part (a) we have a Markovian transition out of state s labeled with rate 2A, 
meaning that the sojourn time in state ,s is exponentially distributed with rate 2A. 
Thus the probability of leaving it within time a is 1 — e~ 2Xa . From s' we have a 
probabilistic transition labeled with r, leading to t\ and with equal probability. 
Note the dashed arrows denote probabilistic transitions. 
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- Part (c) is similar to part (a), in the sense that first a probabilistic transition out of r 
is enabled, followed with a Markovian transition with rate 2A. 

- Part (b) has only Markovian transitions. Starting with state t, the sojourn time is 
exponentially distributed with rate 2A. If the transition is taken, there is a race 
between the transition to t\ and t-2 respectively. The probability that the transition 
to state t\ wins the race is thus i. As a result, the overall probability of reaching 
state t\ within time a is (1 — e~' 2Xa ) ■ i. Note that from t no probabilistic transitions 
can be reached. 

The weak bisimulation defined in flOll . written as « e / lz , identifies s and t: s ~ e hz t. 
Intuitively s ~ e hz t because both s and t will leave their original states after an 
exponential delay with rate 2A, and after leaving s and t they will reach either t\ with 
probability 0.5, or t 2 with probability 0.5. 

However the weak bisimulation distinguishes t and r, i.e., t zfc e hz r - Different 
from s, r will make a probabilistic choice first, and then move to either t\ or t 2 after 
an exponential delay with rate 2A. Thus the difference between s and r is just the order 
of the probabilistic choice and the Markovian transition. If one does not consider the 
intermediate states, but only the probability and time of reaching the states t\ and ti, 
obviously, all of the three states s, t, r are behaving the same. 

In this paper, we propose early and late semantics for Markovian transitions reflect- 
ing the example above. Under early semantics, Markovian transitions are considered as 
a sequence of sojourn time distributions followed with probabilistic choices. The core 
contribution of our paper is the notion of late weak bisimulation, which is obtained by 
interpreting Markovian transitions as a sequence of probabilistic choices followed by 
sojourn time distributions, as illustrated in the example. However, the late semantics is 
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much more involved to define for MA, especially if from state t also other probabilistic 
transitions labeled with a would have been enabled. In that case, under the late seman- 
tics this additional a-probabilistic transition should also have been enabled after the 
probabilistic choices, even after potential internal transitions from t\ or t%. We show 
that late weak bisimulation is strictly coarser than early weak bisimulation. 

Both early and late weak bisimulations are defined over the derived structure of MA, 
namely through Markov labeled transition systems (MLTS), which is introduced by 
Deng and Hennesy in (6). Moreover, they have proposed another notion of weak bisim- 
ulation, denoted by w^/i. f° r MA. The weak bisimulation enjoys the nice property 
of being a reduction barbed congruence iTHl . i.e., it is compositional, barb-preserving 
(simple experiments are preserved) and reduction-closed (nondeterministic choices are 
in some sense preserved). The relationship between ~ e hz an d ~dh is however unclear. 
In this paper we clarify these relationships. We show that the early weak bisimulation 
induced under our early semantics gives rise to the weak bisimulation ~ e hz, as we H as 
Ridh- Thus, the proposed weak bisimulations ~ e hz an d ~<ft agree with each other, and 
are strictly finer than our late weak bisimulation for MA. Since our late weak bisim- 
ulation is defined over the derived MLTS as well, applying a result in J6), even being 
coarser, our late weak bisimulation is a reduction barbed congruence as well. 

Summarizing, the contributions of this paper are as follows: 

- For MA, we propose early and late semantics for Markovian transitions. Based on 
this notion, we propose early and late weak bisimulations. The latter is shown to be 
strictly coarser. 

- We prove that our early weak bisimulation agrees with both the weak bisimulation 
proposed by Eisentraut, Hermanns and Zhang in 0101 . and with the weak bisimula- 
tion proposed by Deng and Hennesy in J6). 

- We propose early and late weak simulations along the same line, and clarify the 
relation to weak simulations proposed in the literature. 

Organization of the paper. Section [2] recalls some notations used throughout the pa- 
per. In section [3] we give the definition of MA as well as its early and late semantics. 
The novel weak bisimulation is proposed with its compositionality being discussed in 
Section|4] In Section|5] we extend the results to early and weak simulations. Section|6] 
we investigate the relations between our weak bisimulations with the weak bisimula- 
tions introduced in ifTUl and JSJ. In Section|7]we briefly discuss how time-divergent MA 
are dealt with previously, and argue that our late weak bisimulation is also the coarsest 
reduction barbed congruence in Section[8] Section[9]concludes the paper. 

2 Preliminaries 

Let S be a finite set of states ranged over by r, s,t, . . .. A distribution is a function 
[i : S — > [0, 1] satisfying p(S) = J2ses M s ) ^ 1- If M^) = 1. it is called a full 
distribution, otherwise it is a sub distribution. Let ADist(S) denote the set of all (sub 

or full) distributions over S, ranged over by Moreover, we use Dist(S) to 

denote the set of all full distributions. Define Supp(fi) = {s | p(s) > 0} as the support 
set of /i. If p(s) = 1, then p is called a Dirac distribution, written as V s . Let \p\ = p(S) 
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denote the size of the distribution fx. Given a real number x, x ■ fi is the distribution such 
that (x ■ fJ,)(s) = x ■ n(s) for each s G Supp(fi) if x ■ < 1, while (j, — s is the 
distribution such that (/i — s)(s) = and (/i — s)(t) = /i(t) with s ^ t. Moreover 
fi = fi\ + fj,2 whenever for each s G (Supp(fii) U Supple)), £*(s) = Mi( s ) + /^(s) 
and < 1. We often write {/i(s) : s \ s G Supp(fi)} alternatively for a distribution 
/i. For instance, {0.4 : si, 0.6 : S2} denotes a distribution /i such that /i(si) = 0.4 and 
A*(*a) = 0.6. 

3 Markov Automata 

In this section we recall first the definition of Markov automata introduced in ifTol . 
Then, we give the early and late semantics of Markov automata in terms of Markov 
labeled transition systems. 

Definition 1 (Markov Automata). An MA AI is a tuple (S, Act T , — > , -», so), where 

— S is a finite but non-empty set of states, 

— Act T = Act U {t} is a set of actions including internal action t, 

— — > C S x ylct T x Dist(S) is a finite set of probabilistic transitions, 

— 5 x R + x 5 « a finite set of Markovian transitions, and 

— So € S is the initial state. 

Let a, (3, 7, . . . range over the actions in Act T , A range over the rates in R + . More- 
over, let a r , /3 r , 7r, ■ • ■ range over Act T U M + . A state s G S is stable, written as s J,, if 
s similarly fi is stable, written as fj,],, iff s4- for each s G Supp{jj). As in 11121101 . 
the maximal progress assumption is assumed, meaning that if state s is not stable, no 
Markovian transitions can be executed. 

Let rate(s, s') = ^2{X \ s -» s'} denote the rate from s to s'. Also the function 
rate is overloaded such that rate(s) = J2 S 'es m ^ e ( s i s ') which denotes the exit rate of 
s. For a stable state s, the sojourn time at s is exponentially distributed with rate equal 
to rate(s), and the probability of one of the Markovian transitions being taken within 
time [0, a] is equal to 1 - e - rate < s ) a . 

MA extend the well-known probabilistic automata (PA) iflTl and interactive Markov 
chains (IMC) lfT2l . Precisely, if the set of Markovian transitions is empty, i.e., -» = 0, 
we obtain PA. On the other side, if distributions are all Dirac, i.e., — > C S x Act T x T>s 
with T>s = {T> s I s G S}, we obtain IMCs. Following [6|, MA will be studied indirectly 
through the Markov labeled transition system: 

Definition 2 (Markov Labeled Transition System). A Markov labeled transition sys- 
tem (MLTS) L is a triple (S, Act T , — >) where S and Act T are the same as in Defini- 
tion^ and — > C S x (Act T U R + ) x Dist(S) is a finite set of transitions satisfying 

s — ^ jUi and s — h [±2 implies that s Ai = A2, and /ii = \i<i. 

Different from the definition of MA, in Definition |2] we require that s — ^ fii and 

s -^h (12 implies that s -5>, Ai = A2, and /ii = (12- This means that each state in 
an MLTS can only have at most one Markovian transition, but after the Markovian 
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transition, it will evolve into a distribution instead of a single state as in MA. This is 
not a restriction, but just expresses the race condition explicitly. In MLTS the maximal 

progress assumption is also embedded in the definition, i.e. s A> fj, implies that s 

As usual, a transition can be lifted to distributions, that is, \i — ^ \i! iff for each 
s e Supp(n) there exists s fj, s such that J2 s eSupp(^) M s ) 1 Ms = 

3.1 Early Semantics of Markov Automata 

Definition 3 (Early Semantics). Let M = (S, Act T ,—>, so) be an MA. The early 
semantics of M is defined as an MLTS, denoted by *M = (<S, Act T ,» — !•), where 

— > C • — > and s • — > /i, iff sj. A A = rate(s) A Vs' G Supp(p,).p(s') = ^ ■ 

In the equation above we require that s is stable as usual due to the maximal progress 

assumption. As an example for the MA in Fig.[TJb), t —t {(^ : ti), (i : ^2)} according 
to the early semantics. 

3.2 Weak Transitions for M LTS 

To define the late semantics for MA, we need the notion of weak transitions which shall 
be introduced in this section. In order to abstract from the internal action of L, we let 
s [i denote that a distribution fi is reached through a sequence of steps which are 
internal except one of which is equal to a r . Formally, the weak transitions for MLTSs 
are defined as follows: 

Definition 4 (Weak Transitions for MLTS). The weak transition relation =^ is the 
least relation such that, s =5> fi iff 

1. a r = t and ji = V s , or 

r r 

2. there exists a step s > /i' such that [i = 2~2s'ESupp(fj,') 

At'(s') • Us', where s' =^> 

/V if j3 r = a r , otherwise s' ===> fj, s i and j3 r = t. 

Intuitively, through the weak transition, s reaches the distribution p, through an 
history-dependent scheduler, very much the way it is introduced in ifTTl . In more detail, 
the first clause says that in case a r = r, we can stop at s. Otherwise, from s the action 
j3 r is chosen leading to the distribution p! , such that: 

- if j3 r = a r , then each state s' in the support of fj,' reaches fi s > only through a 
sequence of r actions, 

- if /3 r = t, then each state s' in the support of [}! reaches /v through a weak 
transition s' ===> fi s i 

Stated differently, we unfold a tree with the root s, the successor states are determined 
by the action chosen from the node. It is history dependent as each state s may occur 
in different nodes in the tree, and each time a different transition may be chosen. We 
say that the weak transition s ==^> p is a deterministic weak transition if in addition 
it satisfies the property that each state picks always the same transition whenever it is 
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visited. In the sequel we shall use s =^>d A* t0 denote deterministic weak transitions, 
which will be used later in defining the late semantics. Note that only finitely many 
deterministic weak transitions exist, see 0. 

The weak transition defined in Definition|4]can be lifted to distributions in a straight- 
forward way as for strong transitions. Equivalently, weak transitions can be formalized 
elegantly using trees as in (8), or using infinite sum Q. The advantage of this definition 
will be clear in proving the equivalence results of all the existing weak bisimulations. 

3.3 Late Semantics of Markov Automata 

When defining the early semantics of an MA in Definition[3] a stable state with Marko- 
vian transition is equipped with a transition labeled with its exit rate A, followed by a 
distribution depending on the race condition. As discussed in the introduction, in the 
late semantics, we switch the interpretation, namely the state first evolves into a distri- 
bution according to the race condition, followed by a Markovian transition labeled with 
A. 

In the late semantics we shall make use of the set of states 

\S\ := {{a, t\ | s, t G S A s| Arate(s) > 0}. 

The outgoing transitions from these new states are defined by: [s, ifl be a state such that 

i) [s, t\ — > t where A = rate(s), and ii) [s, — > /i iff s — > p. Intuitively, [s, t] is a new 
state having exactly the same non-Markovian transitions as s, and can evolve into t via 
a Markovian transition with rate equal to rate(s). Moreover, for a distribution p over S, 
we let [s, jLt] denote the corresponding distribution over [<S] satisfying [s, /i]([s, tfl) = 
p,(t) for all t G S, The late semantics is defined as follows. 

Definition 5 (Late Semantics). Let M = (S, Act T , — >, -», sq) be an MA. Moreover, 
let the MLTS *M = (S, Act T , • — >) be its early semantics. The late semantics of M, 
denoted by Al* = (<SU Act T , — )•), /* the smallest ^MLTS such that for each s G S 

1. s • — > fi implies that s — * p,, 

2. s • — /i implies that s — * [s, //] and for all [s, t] G Supp(fs, fif), 
Is, tj — )• V t and [s, t\ 1/ iff s ^. 

The idea of Definition [5] is to postpone the exponentially distributed sojourn time 
distribution of s after the probability choices. The first case is trivial where all other 

A T 

non-Markovian transitions from s will be then copied. If s • — fj,, then it can be 
seen that p is obtained by applying the race condition after the Markovian transition. 
As a result in the late semantics we can let s choose the successors according to the 
race condition first, and then perform other delayed actions. Therefore s —^m [s,/n] 
where for each Js, t] G Suppds, pf), there exists a t G Supp(p) such that all the 
delayed non-Markovian transition of s is enabled at [s, t\ i.e. [s, ij — ^> v iff s «-^-> i/, 

moreover [s, ij will leave for 2? t via Markovian transition with rate A i.e. [s, tj — )• XV 
Essentially, for each t G Supp(p) and s we introduce a new state [s, t] G [<S] such that 
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all the delayed non-Markovian transition of s and the delayed Markovian transition to 
t are enabled at [s,i]. The following two examples illustrate how the late semantics 
works. 

Example 1. For the MA t in Fig.[TJb), by adopting the late semantics, we have t 

{i : ft, tij, \ '■ Pi ^2]} in the resulting MLTS where the only possible transitions for 

2 A 2A 

[Mi] and lt,t 2 j are [t,ii] — * V tl and \t,t 2 \ — » V t . 2 . Considering the MA in 
Fig.[TJa), since s > XV •==Kd {(| : ii), (| : £2)}, according to Definition [5] we 
will also have s — : [s,ti], \ : [s,^]}, in the resulting late semantics MLTS. 
Thus, the three systems are equivalent w.r.t. the late semantics. Note for states without 
Markovian transitions like r in Fig.Q](c), we do not need to introduce extra states for 
them. 

Example 2. Suppose we have an MA shown in Fig. |2jb). It is not hard to see that 

3 A 

so • — > (J. such that p = {i : S3, | : S4} according to the early semantics which 
is illustrated by the MLTS in Fig. 0a). Instead if we adopt the late semantics, we can 
move the probabilistic choice upward, and thus postpone the execution of other actions. 
Specifically, we allow sq to have a transition sq -^m [so,/i] where [so,/i] = {| : 
[so, S3], § : [so, S4]}, moreover [so, S3] and [so, S4] are two new states where all the 
delayed actions including the Markovian action are enabled i.e. a and 3A in this cased. 
Formally, [so, S3] -^m T> S1 and [so, S4] -—m V Sl because of so •— > D S1 , moreover 

3 A 3 A 

[so, S3] —m T> S3 and [so, S4] — * 7_? S4 because of rate(so) = 3A. The correspondent 
MLTS of so according to the late semantics is shown in Fig.|2c). 

A few remarks are in order: 

1. We have used deterministic weak transitions •=4>_d to define the late semantics. 
Using weak transitions would do the same job, but induces then late semantics 
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with infinitely many transitions. As the deterministic weak transition in Definition 
[5] involves only internal r transitions, the algorithm in O can be used directly for 
constructing the late semantics. The resulting late semantics can have exponentially 
many transitions. 

2. Notice that in Definition we consider each deterministic weak r transition after 
the Markovian transition in the second clause. Indeed, it is not enough to only con- 
sider strong t transitions. Intuitively, by using deterministic weak r transition we 
can postpone the execution of the exponentially distributed sojourn time distribu- 
tion after any probabilistic internal transitions, not just that with one step. Refer to 
Example|6]in the next section for more details. 

3. The size of [5] is in the worst case |tS| 2 . By the definition of late semantics, we only 
need to consider states [s, tj such that s^, and t is reachable from s via r transitions 
after the Markovian transition. Thus, in a real model the size of [5] is expected to 
be much smaller. 

4 Weak Bisimulations 

Before we introduce early and late weak bisimulations, we define some notations about 
transitions for MLTS. For a given MLTS L = (S, Act T , — >), we define -^> p and =5> p 
as following: 

Definition 6. 1. p — ^ p /u' with p G (0, 1] iff there exists a p — p\ + pi such that 
p = I and either a r = t and pi = - • p\, or ~ • p,\ — ^> p! , 

2. p ===> p jjf with p E (0, 1] iff there exists a p = p\ + /i2 swc/z f/iaf p = |/ii| ant/ 



Intuitively, the index p is the part of the distribution of p which makes the move to p', 
which is scaled by - such that pi is a full distribution. Note that the condition "a r = r 
and p! = - ■ pi' in clause 1 of Definition [6] is necessary, refer to the Example[3]for a 

detail discussion. In the following let Suc(p) = {v \ 3p > 0.(p —^p v)} denote the 
successors of v, and Suc*(p) be the transitive closure, called the derivatives of p. 

4.1 Early and Late Weak Bisimulations 

Below follows the definition of our weak bisimulation for MLTSs. 

Definition 7 (Weak Bisimulation). Let L = (S, Act T , — >) be an MLTS. A relation 
1Z C Dist(S) x Dist(S) is a weak bisimulation over L iff p 1Z v implies that 

1. whenever p -^-> p p! , there exists a v =5>p v 1 such that p! 1Z v' , 

2. whenever v —^p V ', there exists a p => p p! such that p! 7Z v' . 

p and v are weakly bisimilar, written as p w L v, iff there exists a weak bisimulation 
1Z such that plZ v. Moreover s k, l r iffV s w L £>,.. 
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(a) 



(b) 



Fig. 3. Two distributions which should not be weakly bisimilar. 



Intuitively, if two distributions p and v are weakly bisimilar, then whenever p is 
able to make a transition labeled with a r with probability p, v must be able to mimic 
the transition with the same probability such that their resulting distributions should be 
weakly bisimilar as well. As mentioned before, the condition "a r = r and p! = - • pi" 
in clause 1 of Definition [6] cannot be omitted, refer to the following counterexample. 

Example 3. Suppose there are two distributions p and v given in Fig. [5] (a) and (b) 
respectively where cn(\ < i < 4) are pairwise different, then if we omit the condition 
"a r = r and p! = - ■ p\" in Definition|6] p only has four strong transitions: p — ^> i 

T>t 1 , p — >i T>t 2 , p — K T>t 3 , and p — >i 2? t4 , each of which can be simulated by v 
and vice versa. Therefore we will conclude that p and v are weakly bisimilar according 
to Definition [7] This is against intuition since p can evolve into s\ with probability | 
where only transitions labeled with ct\ and are possible, this cannot be simulated by 
;/. 

Definition [7] is defined upon MLTSs. For MA, below we shall introduce early and 
late weak bisimulations based on the early and late semantics, respectively: 

Definition 8 (Early and Late Weak Bisimulation). Let M = (S, Act T , — >, s ) be 
an MA. Then, p, v £ Dist(S) are 

1. early weakly bisimilar, written as p ^ v,iffp ps m v, 

2. late weakly bisimilar, written as p v, iff p v. 

In the above definition, we skip the superscript M in and r?, as we assume there 
is a given MA M = (S, Act T , — >, -», so), if not mentioned explicitly, throughout the 
remaining parts. 

Example 4. Recall the example given in Fig. Q] we have shown that s t, but s ^6 r 
since V r can evolve into v via a r transition where v cannot be simulated by T> s or 
any derivative of it. But by considering the late semantics, s will also have a transi- 
tion similar to r, that is, s -^m \s,p\ which is obviously able to simulate v since 
s •^••=^£> p, thus we have ,s r. 
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Fig. 4. Example of late weak bisimilar states. 



Example 5. Suppose we are given an MA where the states to and t' behave following 
the way described in Fig.HJa) and (b) respectively. Then it can be shown that to ~ t' . 

For instance for s 2 inFig.Hfb), since s 2 • — > 9 =>d M= {(§ : s 3 ) , ( ^ : s 4 )} according 
to the early semantics, we have s 2 [^2, A*l according to the late semantics. It is 

easy to check that v w* |s2,/xj. The other cases can be checked in a similar way, 
therefore by Definition |8] to ~ t' . Notice that to ^ M i.e. to ^ t'o> smce v cannot 
be simulated by any derivative of t' - By interpreting t' using early semantics, s 2 can 
only evolve into {(-| : S3), (| : S4)} via Markovian transition with rate 3A. Therefore 

In Definition|5]we consider each deterministic weak t transition after the Markovian 
transition, since it turns out that it is not enough to only consider strong r transition, 
refer to the following counterexample. 

Example 6. Let us consider s and r in Fig. [TJa) and (c) again, if we only consider 
strong r transition in Definition [5] i.e. replacing s • — p in the second clause 

A T 

by s • — >• — > [i, then still s ?a* r. But this does not work in general, for instance if 
we change s a little bit by adding another intermediate state s" such that s' ^> s" and 

s" ^> p, then s » 2A > • r > T> s », thus we will have s [s, T> s it\ where [s, s"J -^-)« 
!D S // . Since s is the only state with Markovian transition in Fig.Q~](a), hence all the other 
states will have the same transitions in the late semantics MLTS. It is not hard to see 
that s tff r according to Definition [8] since neither V ri nor V,. 2 can be simulated by 
any derivative of s, this is against our intuition. 



Late Weak Bisimulation for Markov Automata 



1 1 



4.2 Properties of Early and Late Weak Bisimulations 

In Definition|7]we have used strong transitions on the left side of Clauses 1 and 2. As in 
the standard setting for transition systems, in the lemma below we show that the weak 
bisimulation does not change if we replace the strong transitions by weak transitions. 
This simple replacement is very useful for proving the transitivity, which we shall see 
later. 

Lemma 1. Let L = (S, Act T , — >) be an MLTS. A relation TZ C Dist(S) x Dist(S) is 
a weak bisimulation iff pTZ v implies that 

1. whenever p ===> p /J.', there exists a v ==> p v 1 such that p! TZ v' , 

2. whenever v ==> p V ', there exists a p => p p! such that p! TZ v' . 

The following theorem shows that the weak bisimulation defined in Definition|7]is 
an equivalence relation, and is strictly coarser than ^s. 

Theorem 1. For any MLTS L, m L , *«, and are equivalence relations, moreover 

4.3 Compositionality 

In this section we show that and are congruence relations for time-convergent 
MA. First we recall the notion of time-convergent and time-divergent MA. 

Definition 9 (Time-convergent). A state s is time-convergent iff there exists s =!=> p 
such that p],, otherwise it is time-divergent. Let M = (S, Act T , —¥ , — », sq), then M is 
time-convergent iff for each s G <S, s is time-convergent, otherwise M is time-divergent. 

The reason to distinguish time-divergent and time-convergent states is because of 
the maximal progress assumption, that is, the internal action takes no time and can ex- 
empt the execution of Markovian transitions, thus for a time-divergent state, it will have 
infinite r transitions with positive probability according to Definition [9] as a conse- 
quence it will block the execution of Markovian transitions. 

Now we recall the parallel composition defined in ifTUl as follows: 

Definition 10 (Parallel Composition). Let Mi = (Si, Act T , — > i, s' ) and M 2 = 
(Si, Act T , — >• 2,-»2,s'q) be two MA, then Mi \\a M 2 = (S, Act T ,—> , -», so) such 
that 

- S = {si \\ A s 2 I (si,s 2 ) e Si x £2}, 

- (si \\a s 2 ,a,pi \\a P2) £— > iff either a € A and Si — > pi or a £ A, s; — > pi, 
and pz-i = Pss^j with i £ {1, 2}, 

- (si \\a s 2 ,X,s[ \\ A s' 2 ) iff either 

• Si = s'p Si -» s'p and A = Ai + A2, or 

• Si -» s'i and s' 3 _ t = s 3 _i 
with i G {1,2}. 

- so = s' |U 8%, 
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where fi\ \\a [ii is a distribution such that (/xi \\a M2)( s i \\a S2) = Mi( s i) ■ M2(s2)- 

The theorem below shows that both *« and are congruent with respect to \\a 
for time-convergent MA: 

Theorem 2. For time -convergent MA, it holds that: 

1. (/! \\a Mi) ^ IIa Mi)/ or any fi\ provided that /1 f. 

2. (/i \\a Mi) ^ II A f or a- n y Mi provided that fx 1/. 

The above theorem does not hold for time-divergent MA. A detailed discussion is 
given in Section[7] 

5 Weak Simulations 

In this section we introduce the weak simulations with respect to early and late seman- 
tics respectively. We first give their definitions, and then show their properties. 

5.1 Early and Late Weak Simulations 

Given the definition of weak bisimulation in Definition |7l we can define weak simula- 
tion in a straightforward way as follows: 

Definition 11 (Weak Simulation). Let L = (S, Act T , — >) be an MLTS. A relation 
TZ C Dist(S) x Dist(S) is a weak simulation over L iff fiTZ v implies that whenever 
[i -^r p jf , there exists a v => p v' such that [i! TZ v' . 

Let n and v be weakly similar, written as /i ^ L v, iff there exists a weak simulation 
TZ such that [iTZv. Moreover s ^ L r iffV s ^ L V r . 

As in Section |U we shall introduce two weak simulations based on early and late 
semantics of MA respectively. 

Definition 12 (Early and Late Simulation). Two distributions fi, v over S are 

1. early weakly similar, written as M ^~ v > iff ~M ~ M v > 

2. late weakly similar, written as fi ^ v, iff fi ~ Ai * v. 

Bellow we give a simple example illustrating the early and late weak simulations. 

Example 7. Let s, t, and r be the three MA in Fig.Q] moreover let so be the MA same as 
s except that it has an extra transition: sq — » s' . Then it is not hard to see that t ^ so, 
t ;5* so, and r ^* sq, but r sq does not hold. Since r can evolve into v which 
cannot be simulated by r under the early semantics. 

If we omit the state s' and its related transition in Fig. [T](a), then s and r can be 
seen as the resulting MLTSs by interpreting t according to the early and late semantics 
respectively. As mentioned in Example Q] we have s »^ r. Also note that s r, but 
r *j& s with the same argument as r sq. In other words, by interpreting t according 
to the late semantics we actually preserve the weak simulation. 
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5.2 Properties of Early and Late Weak Simulations 

In this section we will show several properties of the weak simulations. We first prove 
that they are preorders. In order to do so, we introduce the following lemma similar to 
Lemma Q] showing that the weak simulation does not change if we replace the strong 
transitions by weak transitions. 

Lemma 2. Let L = (S, Act T , — >) be an MLTS. A relation TZ C Dist(S) x Dist(S) is 
a weak simulation iff fj, TZ v implies that whenever fi => p [i! , there exists a v => p v' 
such that )a! TZ v' . 

The following lemma shows that the weak simulation for MLTSs defined in Def- 
inition QT| is a preorder for any L, and as in Section |4]~* is strictly coarser than 

Theorem 3. For any MLTS L, ^ L and ^ are preorders, moreover C 

Bellows we show that both *^ and ^* are congruences with respect to the operator 
\\a on time-convergent MA. 

Theorem 4. For time-convergent MA, it holds that: 

1. (fi \\a fJ-i) *~ iy \\a (J- i) for any fix provided that fi *^ v. 

2. (fi \\a Mi) ~" {y \\a for any fix provided that [i ^ v. 

Let TZ^ 1 denote the reverse of TZ, then the weak simulation kernel ^ L n(^ L )~ 1 is 
strictly coarser than ;3 L shown in the following lemma. 

Lemma 3. For any MLTS L, « L C n(^ i )" 1 ). 

As a direct consequence of Lemma|3] it also holds that C (Ji n(ii) -1 ) and 

^ c (^n(^)- 1 ). 

6 Comparing "te, , « c/u . and 

In this section we compare our weak bisimulations with the weak bisimulation ~ e hz 
in IfTUl , and ~dh m defined upon an MLTS. We show that our early weak bisimula- 
tion agrees with both ~ e hz an d ~dh, implying that ~ e h z = ~dh- First, we shall recall 
the definitions of « e fez an d ~dh m the following. 

6.1 Weak Bisimulation a la Eisentraut, Hermanns and Zhang 

In this section we recall the definition of weak bisimulation introduced in flOl . For 
simplicity we do not consider combined transitions here, since all the bisimulations 
defined in this paper can be changed accordingly by taking combined transitions into 
account without affecting the theories. According to Lemma 2 in IfTUl we adopt the 
following definition of which shall be easier for proving the relationship to our 
weak bisimulations. 
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Definition 13. Let L = (S,Act T ,-t) be an MLTS. A relation TZ C ADist(S) x 
ADist(S) is an EHZ-weak bisimulation iff (iTZ v implies that = \v\ and 

1. whenever p => [iP + v s , there exists a v ===> v 9 + v s such that 

- fx 3 TZv 9 and fi s TZ v s , 

— if fi 9 ==> fi', then there exists a v 9 =S> v' such that fi' TZ v' . 

2. symmetrically for v. 

/! and v are EHZ-weakly bisimilar, written as p ~^ hz v, iff there exists an EHZ-weak 
bisimulation TZ such that pTZ v. Moreover s riffD s ~^y iz T) r - 

Intuitively for p and v being EHZ-weakly bisimilar, their sizes must coincide. More- 
over if [i can split into [i 9 and v s i.e. /i => [i 9 + /i s , then \i should also be able to split 
into two parts i.e. v v 9 + v s such that fi 9 ^ hz v 9 and fi s ~^ hz V s . Also if [i 9 can 
evolve into fi' via weak a r transition, in order to simulate it, v 9 is also able to evolve 
into v 1 via weak transition with the same label a r , and their resulting distributions p! 
and v 1 are still EHZ-weakly bisimilar. 

Even though ~^ hz is originally defined on any distributions in 11101 . we can easily 
change it to deal only with full distributions due to normalization: 

Lemma 4. Let p and v be two distributions. Then p ~^ hz v iff \p\ = \v\ and 

According to the lemma above, we shall restrict the discussions to full distributions 
while discussing the relationships between various weak bisimulation relations in the 
following sections. 

6.2 Weak Bisimulation a la Deng and Hennesy 

In J6| another definition of weak bisimulation is proposed but with the definition of 
MLTSs being slightly different. By lifting their weak bisimulation to the MLTSs defined 
in Definition[2] we obtain the following definition. 

Definition 14. Let L = (S, Act T ,-¥) be an MLTS. A relation TZ C Dist(S) x Dist(S) 
is a DH-weak bisimulation if pTZv implies that 

1. whenever p =£>- Pi ■ fJ*i, there exists a v ==^ Pi ■ Vi such that pi TZ Vi 
for each i € I, 

2. whenever v =5> YlieiPi ■ v i> there exists a p ~^2 ieI Pi • Pi such that pi TZ Vi 
for each i <E I. 

where I is a finite set of indexes and YliPi Pi = 1- ^ ef M an< ^ v be DH-weakly bisimilar, 
written as p f«Jl v, iff there exists a DH-weak bisimulation TZ such that p TZ v. 
Moreover a r iffV s wjj, V r . 

Definition [Til and [l4l are defined upon a given MLTS, similar as in Definition|8]we 
can lift them to an MA in a straightforward way. In both 0101 and |6), only the early 
semantics is considered, thus we have the following definition. 
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Fig. 5. Summary. 

Definition 15. Given an MA M = (iS, Act Tl — > 1 -», so) anc/ distributions /i and v over 
S, (J, ~ehz v iff > «"ftf ^ similarly \i K, dh v iff K,' d f v. 

Example 8. Given an MA where s, t, and r are depicted as Fig.Q] by the early seman- 
tics, t has a similar transition as s, and can evolve into distribution [i via a Markovian 

transition labeled with 2A, i.e. t ^ = {± : si, | : s 2 }.LetK = {(V s ,V t ), (D' s ,[x)}U 
ID where ID is the identity relation, it is not hard to see that TZ is both an EHZ-weak 
bisimulation and a DH-weak bisimulation by Definition [T3l and [T4l thus s ~ e hz t and 
& ~dh t. But for r there is no way for s and t to simulate it, for instance n can evolve 
into <i directly via a Markovian transition labeled with 2A, while no state or distribution 
in s and t can do so, thus neither t ~ e h z t nor t ~dh r. 



6.3 s^eftj and ss^ are Equivalent 

In this section we show the relations of all the simulation and bisimulation relations. To 
be clear it is worthwhile to emphasize that the definition of r? is upon the late semantics 
of the given MA, while all the others are defined upon the early semantics. 

Let ^ehz and denote EHZ-weak simulation J9) and DH-weak simulation, whose 
definitions can be obtained by omitting Clause 2 in Definition [T3l and [Irrespectively. 
Below we show that the early weak simulation and weak bisimulation agree with that 
in the literature, respectively: 

Theorem 5. 1. *te = xs ehz = w dh , 



6.4 Summary 

We summarize all the relations in Fig.|5]where — > denotes "implication" while de- 
notes that the implication does not hold. Moreover *«, ~ e hz: an d ~dh are in the same 
node meaning that they are equivalent, similarly for ^ e /iz, and 
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7 Divergence Sensitive (Bi-)simulaions 

We have shown that *« agrees with both ~ e hz an d ~dh- The latter two relations have 
been shown to be congruences with respect to parallel compositions, but only for time- 
convergent MA. The reason why Theorem [2] does not apply for general MA can be 
understood by the following example considered in paper J6) . 

Example 9. Assume that we have two states s, r with s having no transition available 
while r only has a self loop labeled with r. It easy to check that s and r are weakly 
bisimilar according to all the three weak bisimulation definitions. Now consider another 
state t with only a self loop labeled with A. After parallel composition with s and r, 
(s ||^4 t) and (r \\a t) are no longer weakly bisimilar, as the A loop has no effect for 
state r \\a t because of the maximal progress assumption. 

This problem was elegantly solved in lfl2l by adding a third condition for defining a 
divergence sensitive weak bisimulation, that is, two weakly bisimilar states either both 
are divergent or none of them diverges. In this section we discuss briefly that our notion 
of weak bisimulations can be refined such that it reflects the divergence. 

We say the distribution p is time-divergent iff for all s G Supp(p), s is time- 
divergent. Below we present the divergence sensitive weak bisimulation for MA: 

Definition 16. Let L = (S, Act T ,->) be an MLTS. A relation TZ C Dist(S) x Dist(S) 
is a divergence sensitive weak bisimulation over L iff pTZ v implies that 

1. whenever p —^-p p', there exists a v => p v' such that p! TZ v' , 

2. whenever v — ^> p v' , there exists a [i, => p p! such that p! TZ u', 

3. p is tune-divergent iff v is time-divergent. 

[i and v are divergence sensitive weakly bisimilar, written as p ~div v < ff there exists 
a weak bisimulation TZ such that pTZv. Moreover s ~^ iv rijfD s T> r . 

Based on the above definition, early and late divergence sensitive weak bisimula- 
tions can be defined directly on MA. The simulation variants can be obtained by omit- 
ting the second clause of Definition [16] Obviously, the divergence sensitive weak (bi- 
simulation is strictly finer than their corresponding non-sensitive counterparts, and they 
agree with each other for time-convergent MA. Moreover, the compositional results (cf. 
Theorem|2]i holds true for all MA if one consider divergence sensitive (bi-)simulations. 

It is easy to check that according to Definition Q~6] states s and r in Example [9] are 
not divergence sensitive weakly bisimilar. 

8 Related Work and Discussion 

Weak bisimulations have been studied for various stochastic models, for instance for 
Markov chains HI 121 . interactive Markov chains lfl2l . probabilistic automata II15I17I . 
and alternating automata [8|. MA arise as a combination of probabilistic automata and 
interactive Markov chains. Two - seemingly - different weak bisimulation semantics 
have been proposed in [10 6 1 for MA. They have been shown to be equivalent in this 



Late Weak Bisimulation for Markov Automata 



17 



paper, moreover, we have proposed a weaker version - the late weak bisimulation - 
in this paper. Another interesting related work is |[T6l . where Rabe and Schewe have 
shown that finite optimal control exists with respect to reachability probability for MA. 

Recently, Deng and Hennesy (6) have proposed another nice solution to deal with 
compositionality for time-divergent MA, by giving a new semantics for the parallel op- 
eratofl using the notion of indefinite delays associated with transition. These transitions 
are also referred to as passive transitions. For s ||yt t being able to perform a Markovian 

transition — s needs be able to perform -H> and t needs to perform a passive transition, 
or vice versa. Thus the Markovian transition will be blocked by participating compo- 
nent without Markovian or passive transitions. Under this new semantics, ft^ is shown 
to be congruent with respect to all MA. In our previous example we have then s fts<a r, 
and moreover s \\a t ~dh t ||^ t, as s \\a t cannot perform Markovian transitions due 
to the fact that s cannot perform any Markovian transition even with indefinite rate. 

Moreover, in J6) Deng and Hennesy have proved that 1=^/1 is the coarsest relation 
which is a reduction barbed congruence, i.e., it is barb-preserving, reduction-closed, 
and compositional w.r.t. a process language (mCCS) with underlying semantics as a 
MLTS - with extension of passive transitionQ In Theorem Q] we have shown that 
is strictly coarser than ~dh, therefore it seems that ft? should not be a reduction barbed 
congruence. Interestingly, ft? is indeed such a congruence. The reason that ft? is coarser 
than Kidh is because that they are defined upon different semantics: ft? is based on the 
late semantics while is upon the early semantics. Moreover, both semantics are in 
terms of MLTSs. In the proof of Theorem [5] we have proved that « L coincides with 
for any MLTS L. Therefore if we define ft^ft upon the late semantics of a given 
MA M, it will be equivalent to ft? due to = ft^ r . Since AT is an MLTS, thus 

as a direct consequence of (6), is also the coarsest relation which is bard-preserving, 
reduction-closed, and compositional w.r.t. mCCS. 

9 Conclusion 

In this paper we have proposed early and late semantics for MA, in terms of the derived 
model MLTS. Based on it, we proposed early and late weak bisimulations. Our notion 
of late semantics (and weak bisimulations) is inspired by switching the exponential dis- 
tributed sojourn time distribution with probabilistic transition. We show that early weak 
bisimulation is strictly finer than late weak bisimulation. Moreover, we establish the 
relationship between weak bisimulations by Eisentraut, Hermanns and Zhang 1 10 1 and 
by Deng and Hennesy (6), and prove that both agree with our early weak bisimulation. 
Thus, our late weak bisimulation is weaker than all of the other variants. 

A future work is to determine the smallest MLTS corresponding to the late weak 
bisimulations. In our definition the MLTS induced by the late semantics can be of ex- 
ponential size, due to the use of the deterministic weak transitions. It might interesting 
to see whether such exponential complexity is inevitable in the definition. 

3 A slight difference is that in (6) || is considered instead of | a ■ 

4 Our discussion here holds directly for the extension with passive transitions. 
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A Proofs 

A.l Proof of Lemma Q] 

Proof. Let TZ = {(p,u) | p ~ L v}, and suppose that pTZv and p ===> p fx' , we are 
going to show that there exists a v =>„ v' such that p! TZ v' by structural induction. 
According to the definition of => p , there exists p —^ Pl pi =*y Pi and P ~~^P2 

P2 =>p! 2 P2 SUch that Pi ■ Pi + P2 ■ P2 — P and ( £l p El - ■ Pi + El J 2 - ■ P2) = P- Since 

fj, w L i>, there exists v => Pl v\ and v =4> P2 ^2 such that p\ k, l 2/1 and p2 « L ^2- 
By induction there exists v\ ==^ p ' 1 ^ and vi =^- P ' 2 v' 2 such that p[ « L z/( and 
p' 2 k l v 2 , so there exists a z/ => p ^' = • ^ + . ^) suc h that p! w L z/ 

i.e. /1' TZ v' . 

The other direction is trivial since the strong transition is a special case of the weak 
transition. 



A.2 Proof of TheoremQ] 

Proof. We first prove that « L is an equivalence relation. The symmetry and reflexivity 
is easy to prove and is omitted here. We only show how to prove the transitivity. Suppose 
that /ii w L fj,2 and P2 ~ L P3, we need to prove that pi w L fx^. By Definition|7J 
if pi Ri L 112 and P2 ~ L /i3, then there exists two weak bisimulations TZ\ and TZ2 
such that fii IZi \±2 and p 2 TZ2 P3- Let 7?. = {(1/1, vz) | 3z^ 2 -^i 7\Li i>2 A ^2 7?.2 ^3}- 
It is clear that p\ TZ pz, so once we can prove that TZ is a weak bisimulation, we can 
say that p\ ps l p^. Suppose that p\ => p p[, then there exists a P2 => p p' 2 such 
that p[ TZi p! 2 - Since we also have P2 7Z2 P3 where 7Z2 is a weak bisimulation, so there 
exists a /13 ==> p p' 3 such that /ij TZ2 p' 3 . By definition of TZ, we have /x^ 7£ /z 3 , so 7\L is 
a weak bisimulation. 

Secondly, we prove that C ~*. Suppose that *A/ = (5, Act T , • — !>) and 

M* = (5', ^4c£ r , — *). First we show that p'K, v implies p, v. Let TZ = {(fi, v) 
p'te v} U 7?/ where 7^' is the least relation satisfying: 

- (\s, f4, [r, w]) S 7e' with 7? s *te 7? r and 

- (p, v) G 7\L if there exists fi\ TZ v\ and /X2 TZ V2 such that /i = p ■ fii + (1 — p) ■ fi2 
and v = p ■ v\ + (1 — p) ■ V2- 

Then according to Definition|8]it is enough to show that TZ is a weak bisimulation w.r.t. 
M*. Let (p, v) G 7£. We need to prove that whenever ^ — °^» p pi, there exists a =5i p 
1/ such that p! TZ v' . First assume that (p, v) ^ TZ', implying that Supp(p), Supp(v) C 
5. We then consider the following cases: 

1. a r = a € Act. If p — -* p p! , then according to Clause 1 p •— >p yu'. Since 
/i *« v, then there exists a v *==> P v' such that pi 'm v' , therefore there also 
exists v =3«p v' such that p! TZ v' since s p" implies that s p" for each 
s. 



20 Lei Song, Lijun Zhang, and Jens Chr. Godskesen 



2. a r = t. We prove by induction n i.e. the size of Supp(fi). If n = 1 and p = V s 
for some s, then by Definition [5] whenever T> s — //' for some //, we know that 
either 

- s • — > (i , or 

- s • — »-»=^d /i" such that // = [s, //']. 

For the first case, it is similar as Clause 1, and is omitted here. For the second case, 

A T 

since P s z/, there exists a z/ •==>! z/' such that //' *« zv", that is, v •=>• 

z^i •^h>»^=> d z/' where 2? s *rj i/^. According to Definition |5] ^ ==> [z^i,^"] 
where [t/i,z/"J = [r, z/'J for some r G Supp{v\), obviously [s,//'fl 1Z [fi, z/']. 
As a result there exists a z/ =4* z/ = [z^i, z/'] such that p! 1Z v' . When n > 1, 
for some s € Supp(p), there exists a z/ *=^> z^i + Vi such that /Lt(s) = \v\\ and 
2? s (t^t ■ v i)- The following proof is straightforward by induction. 

3. a r = A. This case is impossible, since according to Definition [5] only states in [<S] 
can perform Markovian transitions. 

For the case (p, v) G TV we prove that [s, /ij r? [r, i/J provided that T> s *pz 2? r and 
p *sa za Suppose that [s, //J — -)»p pi , then it must be the case that V s • — > pi . Since 
V s "w D r , there exists a D r •==> z/ such that pi *« z/, so we have [r, i/] =S» P z/ 
such that /x' 7?. z/. If [s, /LtJ — then ^z • T > p p! . Since /i *f« ^, there exists a 

•==>p v' such that // •« v' , thus we have [r, v\ =4# p z/ such that p! 1Z v' . This 
completes the proof. 

For the counterexample of r? = ^s, refer to Example|4] 

A.3 Proof of Theorem^ 

Proof. We only prove Clause 1 since the proof of Clause 2 is similar, and can be ob- 
tained in a straightforward way by considering AJ* instead of *M. The proof strategy 
for this result follows the standard way. Let M be the given MA and 'M be the resulting 
MLTS according to the early semantics in Definition[3] We first define the relation 

TZ= {(p \\a V\,v \\a Mi) I ~' M v /\Hi e Disi(5)} 

then, it is sufficient to show that 1Z is a weak bisimulation. Let (/^fn) € ^ with 
po = p \\a pi and Uq = u \\a Pi- Moreover, let po ===>p ^ We need to prove that 
there exists a u ==>p v' such that p' Q 1Zv' . 

Suppose that Supp(p) = js.; \ i G I}, Supp(v) — {s'j \ j G J}, and Supp(pi) = 
{tk | fc G K} where /, J, and K are three finite index sets, then Supp(po) = {sj ||a 
ifc | i G I A fc G A'} and Supp(v ) = {s^ \\a t k \ j G J A k G A"}. The analysis 
of the compositional distribution requires some attention, thus we discuss first different 
cases needed for the weak transition /iq ==^> p /j,' . Whenever /io ===> p p' , then we know 
there exists a set of states C C Supp(po) such that po(C) = p and r => ^ r for each 
r G C where p' = J2 rtEC r ^ ■ p r - While the case a r G A is more clear, the other 
case when a r £ A is a bit more involved. Let r = s, ||a for some t G / and fc G A, 
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so if r =^> p r with a r £ A, then either Si =5> p s and tk =£■ Pt, or Sj /i s and 
tk ==> Pt such that p s \\a Mt = Mr- As a result it is not simple if it is possible to prove 
only by structural induction, instead we need to prove by induction on structure and on 
the size of Supp(p) simultaneously. There are several cases we need to consider. 



1. a r A. 

Suppose that p is a Dirac distribution, that is, p = T> s for a s, then there exists a 
pi =!=> p\ + /if such that pa = (T> s \\a Mi) + (£> s \\a A 4 !)- Moreover we also have 
|4t|- ■ (V s \\ A /if) ^> P1 p s \\ A p 9 2 where V s ^> p s and y^- • p{ =^> P2 /if, and 

TT^T ' {T>s II a Mi) =^p 2 (m' s II a Ma) where p s =^ Ms and j^y ' Mi Ml 
such that p = p x + p 2 and ^ • (p s \\ A p 2 ) + f ■ (p' s \\a Ml) = Mo- In other words 
we can divide po into two parts: V s \\a Mi an d ||a Mi where in V s \\a Mi 
the action a r is performed by T> s while in V s \\a Mi it is performed by /if. Now 
we can use the structural induction. Since p «* M z^, whenever p =^> /i' i.e. 
/i =^>i /i', there exists a v =^i ;/ such that // « M z/, so the following proof is 
straightforward by structural induction. 

Suppose now that the support of /i contains more than one element, then there exists 
a p =^=> p 9 + p s such that p = (p 9 \\a Mi) + (m s ||a Mi)- Since /i m' M v, 
then there exists a v =^=> v 9 + v s such that p 9 « M v 9 and p s k M v s . Also 
for p p' , there must exist y^- • (m 9 IIa Mi) =%i Mo- and T^f ' (m s IIa 

Mi) ==^P2 Mo sucn tnat /° = Pi + P2 and ^ • p 9 + ■ p$ = p' . Since p 9 and p s 
contain less elements in their support than p, we can apply our induction hypothesis 
on them, and the following proof is trivial and omitted. 

2. a r e A. 

As in the first case we first suppose that p is Dirac distribution such that p = 
V s for a s. Then there exists a p\ p\ + p\ such that p = \p\\ and po = 
{V s \\ A M?) + {V s \\ A Mi)- Moreover ^ ■ (D s || A /if) ^ p where £> s ^ 

M s > Tjpr ' Mi =^ mI> and Mo = Ms II A M2- Intuitively, we divide po into two 
parts: 2? s \\a /if and 2? s |U Mi where the synchronization only happens between 
s and Supp(p\). Note that we can do such division only because that p is a Dirac 
distribution, otherwise we cannot always divide po in this way, because each state 
in Supp(p) is not necessary to synchronize with the same set of states in Supp(pi). 
Since p w M v, the following proof is straightforward by structural induction. 
The case when p is not a Dirac distribution can be proved similarly as the first case, 
and is omitted here. 

3. a r = A. 

Again we first consider the case where p = V s for a s. Then there exists a p\ =^=> 
/if + p\ such that p = |/if | and po = (T> s \\a Mi) + « IIa Mi)- Moreover 
|4r|- • (D s ||a Mi) Mo where either i) 2? s p' s , • p\ =h- p% and 

Mo = ^ ' (4 IU (^7 ■ Mi)) + ^ • || Ml), ii) ^ =^ Mo^ • Mi A a «d 
1^- • /if /if such that p% |, or iii) • /if /i , 2? s j=3>, and D s ^ 
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such that p! s |. The following proof is straightforward by structural induction. The 
case when Supp(p) is greater than 1 is similar with the first case and omitted here. 

A.4 Proof of Lemma |2] 

Proof. The proof is similar with the proof of Lemma[T] Let 1Z = {(fx, u) \ p v}, 
and suppose that plZv and p ==^> p p', we are going to show that there exists a v =5> p 
v' such that p! \—n v' by structural induction. According to the definition of ==$- p , 
there exists p -^> P1 p\ =^p' 1 Mi and A* ~~^P2 M2 ==>p' 2 M2 sucn tnat Pi ' Pi + /°2 " 
p' 2 = p and ( Pl p Pl • + P2 p P2 • yu 2 ) = ^. Since p ;3 L 1/, there exists v ==> Pl v\ 
and v => P2 ^2 such that pi ^ L ^1 and /i 2 ~ L ^2- By induction there exists 
v\ ==> p i v[ and V2 ==> P ' v' 2 such that p[ ^ L v[ and /i 2 ^ L z/ 2 , so there exists a 

1/ ^ p 1/ = ■ ^ + ^ • i/ 2 ) such that M ' ^ L 1/ i.e. /i' ft v' . 

The other direction is trivial since the strong transition is a special case of the weak 
transition. 

A.5 Proof of Theorem 

Proof. We first show that ^ L is a preorder. The refiexivity is easy to prove and is omit- 
ted here. We only show how to prove the transitivity. Suppose that p\ P2 and 
P2 ~ L A*3, we need to prove that pi ^ L p 3 . By Definition [TU if p\ P2 and 
/12 ^ L /i3, then there exists two weak simulations 1Z\ and IZ2 such that pi fti /12 and 
/i2 IZ2 A*3- Let 7?. = {(vi, ^3) | 3^ 2 -^i 7?-i A 1/2 7^2 ^3}- It is clear that p\ 1Z p 3 , so 
once we can prove that 1Z is a weak simulation, we can say that p\ ;5 L ^3. Suppose 
that p\ =^> p fx'i, then there exists a P2 ==>p A*2 sucn that p[ 1Z\ p' 2 . Since we also 
have p,2 TZ-2 P3 where IZ2 is a weak simulation, so there exists a p% => p p' 3 such that 
p 2 TZ-2 /^3- By definition of 1Z, we have p[ 1Z p' 3 , so 1Z is a weak simulation. The proof 
of and ^" being preorders is straightforward, since both *M and AJ* are special 
MLTSs. 

The proof of C ^* is similar as Theorem[T]and is omitted here. Intuitively, 
according to Definition [5] we can defer the execution of all the Markovian transitions, 
thus the order of the Markovian transitions and internal transitions does not matter in 
MLTS. 

A.6 Proof of Theorem!] 

Proof. The proof is similar with the proof of Theorem [2] we only sketch the proof 
here. First we assume that p is a Dirac distribution i.e. its support only contains one 
element, then we analysis by cases depending on i) whether p and pi synchronize with 
each other or not, ii) whether the transition is a Markovian transition or not. Then we 
can extend the proof to the case where p is not Dirac, the proof is by induction on the 
number of elements in Supp(p). 
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A.7 Proof of Lemma |3] 

Proof. We omit the parameter L through the proof. The proof of w C n 
is trivial and omitted here. To show that n is strictly coarser than s», it is 

enough to give a counterexample. Suppose we have three states s±, S2, and S3 such that 
s i ~ s 2 ~ S3 but S3 ^ S2 ^ Si. Let s and r be two states such that L(s) = L(r). 
In addition s has three transitions: s — > V S1 , s —> V S2 , s — > X> S3 , and r only has two 
transitions: s —> V Sl , s — > P S3 . Then it should be easy to check that s ^ r and r ^ s, 
the only non-trivial case is when s — > T> S2 . Since S2 ~ S3, thus there exists r V Sa 

such that V S2 C-< D S3 . But obviously s 56 r, since the transition s — > X> S2 cannot be 
simulated by any transition of r. 

A.8 Proof of Lemma |4] 

Proof. First we show that /i v implies = and (jK- ■ fx) ~^j lz (t^--v). 

The fact that \p\ = \v\ is trivial from Definition Qj] Let 72 = {((■njr ' A*) , (y^y- • 
z/ )) I A* ^i/iz ^i' we ^ g° m 8 to P rove that 72. is an EHZ-weak bisimulation. It is 
obvious that \(yK- ■ p)\ = \(tK- • v)\ , and Supp(p) = SuppUjK- ■ p)). For each 

Ml \ u \ IMI 

£ G Supp((-rK- ■ /i)), we also have t G Supp(p). Since psf^ z jy, there exists a 

i/=^i/» + i/» such that i)(jt(t).D t ) i/» and (/*-*) f*,ii) (p(t)-T> t ) 
//, then there exists a ^ 9 1/' such that // ps^, 1/. Therefore there exists a 

(r • v ) ^ ((r- ' ^ 9 )) + ((r • ^ S )) such that (r • M*) • p *) ^ (r • v9 ) 

and • M - <) ~L • ii) (((^ • /*)(*)) ' T>t) ^ (j^ ■ A then there 
exists a (tA- • i^ 9 ) => (■A- • v') such that • //) 72. ■ i/), so 72. is an EHZ-weak 
bisimulation. 

The proof of the other direction is similar and omitted here. 
A.9 Proof of Theorem[5] 

Proof. We only prove the first clause since the other one is similar. We first prove that 
= Rj^, it is enough to show that « L = ~^ hz according to Definition [8] and 
[l5]for any MLTS L. First we prove ^ hz C w l . Let 1Z := {(p, v) \ p ^ hz v), 
then it is sufficient to show that 72 is a weak bisimulation according to Definition|7] For 
each p ==^> p p', we need to prove that there exists a v ==> p v' such that p! 72 v 1 . By 
definition of => p , there exists a p =!=> p 9 + p s such that \p 9 = p and (j^yy ■ A* 9 ) 
//.Since// r^, 1/, then v ^ v 9 + ^ s such that • /i 9 ) • and 

(pq--/i s ) «4z f|4|--^)bvDefinition[T3landLemmal4l In addition fpq--;/ g ) =^ i/ 

such that fx' ~^ hz v', thus /i' 72 v' . As a result there exists a ^ => p v' such that 
/i' 72 v' , so 72 is indeed a weak bisimulation. 

For the other direction we prove w L C ~^ 2 - Similarly we need to prove that 
72 := {(//, v) I /j « L v} is an EHZ-weak bisimulation. Suppose that pTZv and 
/i p 9 + p s , then we first prove that there exists v =^=> v 9 + v s such that 
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1. (j+y ■ p 9 ) K ■ v°) and • p s ) ft (^- • ^), 

2. whenever ( j^-p • p 9 ) /i' 9 , there exists a (u^rr • ;y9 ) v ' 9 such that // 9 7?. z/ 9 . 

If /i =5=> p 9 + /i s , then ===>p (t^t ' M 9 ) with p = |/i 9 | . Since p « L ^, then there 
exists a weak transition v => p (yky ' such that (rjrr ■ A* 9 ) ~ L (rjsy ' 1/8 )> thus 
(jji-p-/i 9 ) 1Z [y^y-v 9 ), then the second clause is easy to verify. It only remains to prove 
that (-Arr ■ A* s ) TZ (t^t ■ Suppose it holds that (ji- • ^ L (ji- • i/ s ), there 
must exist (y^rp ■ Af s ) ==>p A t/S suc h that there does not exist {y^y ■ =^> with 
^/s ^/s gy definition of ===> p , we have p =^> p < pi where p' = \p s \ ■ p and p! = 
p' s , so there exists a v ==> p i v' such that p! « L i/ but i/' ^ v' s . As a result it must 
holds that {y^y-v 9 ) =^> Pl v 1 and ( yjkyv 8 ) =% 2 ^2 such that p r |^ 9 | +p 2 -\v s \ = p' 
and( Pl 'j^ 9 ■v\ + p2 ' p v , ■ -^2) — ^ Since (r^j--/i g ) ~ L (y^--^ 9 ), there exists a weak 
transition (|^- • ,u 9 ) =^ pi /ii such that p 1 « L z^, so we have p =^(| M 9| . pi +| M «| - P ) 

( 1 „ J M I ,'f 1 ., — ■ p' 9 + 1 sT-^-TT^n — • M /s ) which cannot be simulated by v, and this 

Mm 9 I -pi+Ip -p r Im 9 I -pi+Ip s I -p r ; J ' 

contradicts with the assumption that p « L v, thus (-nprr ■ A iS ) ^ (jIT^f ' pS )- By 
Lemma|U 7?. is an EHZ-weak bisimulation. 

Secondly, we show that = P3<#j. As in the proof of = « e fcz> it is enough 
to prove that rs l = for any MLTS L. We first show that p ttj h v implies that 
p « L v. We need to prove that 1Z := {(p, v) \ p v} is a weak bisimulation. 

Suppose that p TZ v and p =^> p //, there exists a ==^ p f' such that pi TZ v' . By 
definition of ==^> p , there exists p ===> p 9 + p s = (\p 9 \ ■ y^y ■ p 9 + \p s \ ■ y^y ■ p s ) 
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such that p = \p 9 \ and - ■ p 9 =^> p! . Since p v, there exists a v =!=> (\v- 

j±y -v 9 + \v s \ ■ j±y ■ v s ) = v 9 + v° such that \p 9 \ = W 9 \ ,j±yV 9 ^ ■ ^ 9 
and j^pj- ■ p s ~^ t^|t- ■ j/ s , so there exists a t^j- • v 9 ^> v 1 such that pi 1/'. 

Therefore there exists a ^ =5> p 1^' such that p! Hv' . 

Secondly we show that p « L v implies that p «jl by proving that 1Z := 
{(p, v) I /i w L v} is a DH-weak bisimulation. Suppose that p 1Z v and p 
Hie 1 Pi ' Mi' then we need to show that there exists a v ==> HieiPi ' such that 
/Uj 7?. for each i £ I, We prove by induction on the size of /. The case when |/| =1 
is simple and we assume that \I\ > 1. Since p ==^ ^2 ie jPi • Pi, there exists a p 
p 9 + p s such that \p 9 \ — p\ and \p s \ — Si/iAiGJ^*' ^ n Edition • /.i 9 /ii 
and j-^j- • j-^j- • X^i^iAie/P* ' Therefore by the similar argument as in 

the proof of •« = ~ e ; lz , there exists a ^ ^=^> ^ 9 + v s such that |/.i 9 | = \v 9 \ , 
t-4-t • u 9 « L t-4-t • and j-^rr • U 3 ~ L ttt ■ V s . By induction there exists 

^ • v 9 ^ v x and ^ ■ ^ s ^> ^ • £^ia*g/P* • ^ such that p. t k l Vi for 

each i G /, so there exists a ^> Sis/ P* ' ^ sucn mat M» ^ ^ ^ or eac ^ ' e ^- This 
completes the proof. 



